Characterization of Real Properties Using 3d Vision

Accurate and realistic 3-dimensional models of urban environments are increasingly important for applications like virtual tourism, city planning, Internet search and many emerging opportunities in the context of “ambient intelligence”. Applications like Bing-Maps or Google Earth are offering virtual models of many major urban areas worldwide. While initially these data sets support visualization, they are inherently capable of addressing a broader purpose. On the horizon are urban models that consist of semantically interpreted objects; an urban 3D visualization will be computer generated, with a fundamental advantage: the urban models can be searched based on object classes. This characterization is the perquisite for a property valuation that we want to enable by semantically interpreting these models. Oblique aerial photography and LiDAR have become a widely used resource for urban imaging and modeling purposes. Originating in the US and championed by Pictometry, oblique images are now being acquired world-wide. We are interested in a comparison between oblique and vertical aerial photography, especially addressing the facades in urban areas and facade details such as the number of floors and windows. Our results show that vertical imagery is wellsuited to facade analysis, and that oblique images deliver results compromised by occlusions. This indicates that the benefit of oblique images is questionable in cases where high overlapping vertical images exist. We are also interested in a comparison between LiDAR data and vertical aerial images, especially addressing the roofs in urban areas and their characterization. We show that the photogrammetric accuracy compares well with LiDAR, yet the density of surface points is much higher from images. This project presents a framework which specifies the processing steps that are necessary for a reasonable semantic interpretation. Our focus is on characterizing the individual buildings on a real property. A key to success in this task is the availability of aerial photography at a greater overlap than has been customary in traditional photogrammetry, as well as a Ground Sampling Distance (GSD) exceeding the traditional values. We first describe the different source data which have to be brought into a common coordinate system. In this process, we build an integrated geometry and semantic object data set that can be analyzed for various purposes. We start out by merging the aerial imagery with the cadastral information to define each property as a separate entity for further analysis. The cadastral data may also contain preliminary information about a building footprint. In cases of missing cadastral data, we also present a novel method for facade separation from aerial images with a success rate of 88%. An important aspect on each real property are its buildings. Therefore, in the next step the building footprints get refined vis-à-vis the mere cadastral prediction, using an image classification and the definition of roof lines. 3-D facade coordinates are computed from aerial image segments, the cadastral information, and the DTM. 8 0 ABSTRACT DOCTORAL RESEARCH: PUBLICATIONS We describe how one can recognize and reconstruct buildings and their facades in 3 dimensions with the purpose of extracting the building size, its footprint, the number of floors, the roof shapes, the number and size of windows, and the existence or absence of balconies. Since facades are often not simply vertical planes, but have depth in the form of stairwells, balconies, awnings or decorations, we need to consider a 3D approach. Mapping a vertical object such as a facade by means of vertically looking aerial photography requires one to cope with very steep look angles and pixels far from square in object space. We can conclude that current high overlaps in imaging campaigns support the creation of useful 3D point clouds of vertical facades. We achieved accuracies sufficient to count windows and floors, even when the buildings had complex shapes with stairwells or balconies. In a test area in Graz (Austria) with 216 buildings, we found that planar facades could be interpreted with success rates between 86% and 93%. When dealing with complex facades the problem gets more difficult. Without the use of the 3rd dimension none of these facades could be interpreted correctly. By including this information, 86% of all floors and 80% of all windows could be counted correctly. However, our focus not only lies in describing the facades of a building but also its roof. Building roofs have, in the last couple of years, been studied with aerial LiDAR point clouds (Light Detection And Ranging). However, recent progress in digital aerial cameras has rendered possible the acquisition of very dense point clouds from high overlap digital aerial imagery, and to use these point clouds jointly with the image information to generate 3D building models. This project presents a multi-step processing framework and work flow for the automatic segmentation and 3D reconstruction of building roofs in densely built-up areas from highresolution vertical aerial images. We are interested in two major topics: the determination of the roof shape and the detection of superstructures like dormer windows, chimneys and skylight to determine the degree of extension of a rooftop. For the determination of the roof shape details extruding from, or intruding into, a roof is being excluded so that each roof is being modeled by means of its planar segments and can then be classified as a specific roof type from a set of standard roof shapes. We show that the results from aerial photography compete well with LiDAR-results as reported by LiDAR researchers. We evaluated three different plane detection techniques and compared their results. In real city data with a total of 1069 superstructures the proposed automated process finds 1015, thus achieving success at a rate of 95%. For our Graz and Annecy dataset with 216 respectively 35 buildings with 810 and 159 roof planes, results in correct roof planes in 95% and 91% of all cases. Small roof structures do confuse the analysis and must therefore be detected and eliminated. These structures can be detected with success rates of 95% respectively 86% of all the test cases depending on the accuracy of the range data. Roof types get classified correctly at a rate of 85%. We also show that we can reconstruct the building roofs and their superstructures such as chimneys and dormer windows with high accuracy and completeness because of the high point density of 100 pixels per m2 in contrast to 25 points per m2 using conventional LIDAR sensors. Concerning the determination of the degree of extension of a roof we achieve a detection rate of 87%. We demonstrate in this project that vertical aerial images are a useful and economical alternative to LiDAR and oblique aerial images for the characterization of building facades and DOCTORAL RESEARCH: PUBLICATIONS 9 roofs. Our experiments are evaluated on a variety of datasets (Graz, Annecy). Our experiments demonstrate robustness and high geometric accuracy of the proposed methods and show that it is feasible to use this information for valuation purposes. 10 0 KURZFASSUNG DOCTORAL RESEARCH: PUBLICATIONS